ENH: Validate AsOfJoin tolerance and attempt interval unit conversion #1952
Conversation
|
hopefully #1963 can get in ahead of this, you can rebase on top of it, and then only your enhancement will be there, not the miscellaneous formatting changes too |
Codecov Report
@@ Coverage Diff @@
## master #1952 +/- ##
=========================================
Coverage ? 86.12%
=========================================
Files ? 93
Lines ? 17116
Branches ? 2164
=========================================
Hits ? 14741
Misses ? 1968
Partials ? 407
|
|
@emilyreff7 please rebase on top of master and then i'll approve! |
Just rebased, looks like only my changes remain :) |
|
these seem like spurious failures, maybe try amending your commit and force pushing to kick off CI again |
looks good now! |
nvm, accidentally closed so the tests are rerunning now :( |
okay, now all good |
Unlike other operations, not all of the arguments of AsOfJoin are automatically passed through the validation chain that occurs in superclass Annotable, since they are not passed into super(). The result is that whereas other operations with offset-like arguments (e.g. Lead/Lag) have their offsets' dtypes automatically inferred, the same does not occur for 'tolerance' in AsOfJoin. We can simply wire this validation in ourselves. Additionally, the default unit for Interval is 's' regardless of the unit of the value of origin. We can make a better attempt at inferring unit before immediately defaulting to seconds. Author: Emily Reff <emily.reff@twosigma.com> Closes ibis-project#1952 from emilyreff7/hackday and squashes the following commits: 2d7ff71 [Emily Reff] Merge branch 'master' of https://github.com/ibis-project/ibis into hackday 40436e1 [Emily Reff] break up tests 134caf3 [Emily Reff] fix failing tests 91318be [Emily Reff] fix failing tests c524401 [Emily Reff] fix failing tests 97fa826 [Emily Reff] validate AsOfJoin tolerance and attempt interval unit conversion
Unlike other operations, not all of the arguments of AsOfJoin are automatically passed through the validation chain that occurs in superclass Annotable, since they are not passed into super(). The result is that whereas other operations with offset-like arguments (e.g. Lead/Lag) have their offsets' dtypes automatically inferred, the same does not occur for 'tolerance' in AsOfJoin. We can simply wire this validation in ourselves.
Additionally, the default unit for Interval is 's' regardless of the unit of the value of origin. We can make a better attempt at inferring unit before immediately defaulting to seconds.